Visual Event Perception

This paper presents a novel framework for training models to recognise simple spatial-motion events, such as those described by the verbs pick up, put down, push, pull, drop, tip, and tap and classifying novel observations into previously trained classes. Simple colourand motionbased segmentation and tracking techniques are used to produce a time series of feature vectors constructed from the 2D object positions, orientations, shapes, and sizes. Hidden Markov models are trained on this time series data and used to classify novel occurrences into previously trained classes. The particular choice of features used allows the system to construct meaningful semantic representations of the event classes that it has learned.